Decoding Coattention Encodings for Question Answering
نویسندگان
چکیده
An encoder-decoder architecture with recurrent neural networks in both the encoder and decoder is a standard approach to the question-answering problem (finding answers to a given question in a piece of text). The Dynamic Coattention[1] encoder is a highly effective encoder for the problem; we evaluated the effectiveness of different decoder when paired with the Dynamic Coattention encoder. We found that models leveraging global attention[2] and locally window-based approaches proved effective, but that a simple decoder combining LSTM neural networks, a linear transformation layer, and a post-decoding length-optimized maximum likelihood layer was the most effective.
منابع مشابه
Deep Coattention Networks for Reading Comprehension
Machine reading comprehension of text is an important task in Natural Language Processing. A recently released dataset, the Stanford Question Answering Dataset (SQuAD) formulates the problem as question answering, and it provides a large corpus of challenging, realistic questions. To address this task, we implement an end-to-end neural encoder/decoder model. The encoder consists of the coattent...
متن کاملQuestion Answering System using Dynamic Coattention Networks
We tackle the difficult problem of building a question answering system by building an end-to-end recurrent neural using network sequence-to-sequence model. We use the coattention encoder and explore three different decoders: linear, single layer maxout, and highway maxout network. We train and evaluate our model using the recently published Stanford Question and Answering Dataset (SQuAD). Out ...
متن کاملDynamic Coattention Networks For Question Answering
Several deep learning models have been proposed for question answering. However, due to their single-pass nature, they have no way to recover from local maxima corresponding to incorrect answers. To address this problem, we introduce the Dynamic Coattention Network (DCN) for question answering. The DCN first fuses co-dependent representations of the question and the document in order to focus o...
متن کاملReading Comprehension with Coattention Encoders
We present an end-to-end network architecture for question answering task given a real-world paragraph context. Using SQuAD dataset composed of 100K pairs, we train a coattention encoder to obtain a knowledge representation of the question and context data. This in turn is given as input to the decoder, which generates answer spans within the context. The prediction ...
متن کاملDCN+: Mixed Objective and Deep Residual Coattention for Question Answering
Traditional models for question answering optimize using cross entropy loss, which encourages exact answers at the cost of penalizing nearby or overlapping answers that are sometimes equally accurate. We propose a mixed objective that combines cross entropy loss with self-critical policy learning. The objective uses rewards derived from word overlap to solve the misalignment between evaluation ...
متن کامل